The present invention generally relates to an image coding system converting apparatus, an image coding system converting method, and a recording medium for recording a program used to execute this converting method. More specifically, the present invention is directed to such image coding system converting apparatus, image coding system converting method, and recording medium, capable of converting a compression coding system while reducing a necessary calculation amount and suppressing a deterioration of an image quality.
Recently, electronic appliances capable of processing digital picture (image) signals are increased, and the digital picture signals used in these electronic appliances are compressed/coded in accordance with certain coding systems (will be referred to as an xe2x80x9cimage coding systemxe2x80x9d hereinafter). There is, however, no compatibility among different image coding systems from each other. Thus, a bit stream coded in one electronic appliance cannot be decoded by such an electronic appliance which performs a coding/decoding operation by using another image coding/decoding system, so that this coded bit stream cannot be reproduced by the latter-mentioned electronic appliance. For instance, when coded picture data which have been stored in a broadcasting station are broadcasted, the following needs may be sometimes made. That is, picture data which has been coded based upon one image coding system different from another image coding system used for the stored picture data is wanted to be decoded for a broadcasting purpose.
To satisfy such a request, the following compressing/coding system is required. That is, a bit stream which has been coded in accordance with a first image coding system is decoded, and this decoded bit stream is again encoded in accordance with a second image coding system. Normally, if the decoder adapted to the first image coding system is combined with the encoder adapted to the second image coding system, then it could realize such a converting apparatus capable of performing the above-explained picture data converting operation.
The major system related to the compressing/coding systems used in the image coding systems for digital picture signals is the hybrid type compressing/coding system. In this hybrid type compressing/coding system, the reduction of the spatial redundancy within the block performed by the DCT (discrete cosine transform) is combined with the reduction of the temporal redundancy between the frames performed by the motion compensation. Furthermore, the data is compressed by employing the entropy coding system. In an actual case, this coding system has been introduced into all of MPEG-1 used in a video CD and a video clip; MPEG-2 used in a DVD and a digital broadcasting system; ITU-T recommendation H.261 used in a television conference/television telephone; ITU-T recommendation H.263 used in a television telephone operable in an analog line; and also MPEG-4 expected as the moving picture coding system for the next generation. In this coding system, DCT, quantizing, motion-compensating, and variable-length coding operations are carried out.
As a result, in the system conversion among these coding systems, the structures of the actual bit streams are resembled to each other. Therefore, while the bit stream coded in accordance with the first image coding system is decoded, the bit amounts every picture and macro-block are counted, so that the process amount required for the bit rate control and the like when the decoded bit stream is coded in accordance with the second image coding system can be reduced.
For example, Japanese Patent Disclosure No. Hei-9-512410 discloses xe2x80x9cMETHOD FOR CODE-CONVERTING CODED VIDEO SIGNAL AND CODE CONVERTING APPARATUS WITH USING CODE-CONVERTING METHOD THEREOFxe2x80x9d so as to reduce the process amount. Now, this code converting apparatus 2000 will be simply explained as prior art with reference to FIG. 17.
FIG. 17 is a block diagram for simply representing the structural arrangement of the code converting apparatus described in Japanese Patent Disclosure No. Hei-9-512410. This code converting apparatus 2000 is arranged by employing the decoding sub-assembly 2001 and the coding sub-assembly 2002. The decoding sub-assembly 2001 contains a demultiplexing circuit 2003 having a function for subdividing the input bit stream into the respective layers, and-the normal decoder 2004. Also, the coding sub-assembly 2002 contains the frame memory 2005, the delay circuit 2006, the normal encoder 2007, the bit rate control circuit 2008, the target bit calculator 2009, and the output buffer 2010.
The decoder 2004 decodes the input bit stream which has been decoded, and contains the variable length decoding means, the dequantizing means, the inverse-DCT means, and the like. Also, the encoder 2007 encodes the non-compression image data, and contains the variable length coding means, the quantizing means, the DCT means, the frame memory, the motion compensating means, and the like. Also, both the bit rate control circuit 2008 and the target bit calculator 2009 determine the quantizing steps in the actual coding operation by referring to the bit amount of the output bit stream. It should be noted that the code converting apparatus 2000 is not equipped with a pre-analyzer having a similar circuit arrangement to that of an encoder. This pre-analyzer encodes an input image at a quantizing step preceding to the step for encoding the input image by the encoder.
Next, the operations of the code converting apparatus 2000 will now be simply explained.
The demultiplexing circuit 2003 separates the input bit stream into the coded data and the overhead other than this coded data, for example, the header information and the start code. Both the coded data and the overhead are supplied to the decoder 2004 and the target bit calculator 2009, respectively. In the decoder 2004, the supplied data is decoded, and the non-compression image data is sequentially written into the frame memory 2005. The target bit calculator 2009 counts the bit amount xe2x80x9cBxe2x80x9d of each of the sub-images such as the macroblocks from both the data and the overhead. Next, the counted bit amounts xe2x80x9cBxe2x80x9d are added to each other over the entire image so as to acquire the bit amount xe2x80x9cSxe2x80x9d of one image. Furthermore, the activity xe2x80x9cXxe2x80x9d of this image is obtained by multiplexing a certain quantizing step xe2x80x9cQ2xe2x80x9d by the bit amount xe2x80x9cSxe2x80x9d of one image.
The target bit calculator 2009 contains the memory, and calculates the bit xe2x80x9cTxe2x80x9d of the present frame based upon the activity xe2x80x9cXprexe2x80x9d of the preceding frame and the obtained activity xe2x80x9cXxe2x80x9d. The target bit xe2x80x9cTxe2x80x9d is distributed to the local target value xe2x80x9cTnxe2x80x9d every macroblock by the distributing circuit provided in the target bit calculator 2009, and then the local target value xe2x80x9cTnxe2x80x9d is sent to the bit rate control circuit 2008. The bit rate control circuit 2008 calculates the quantizing step xe2x80x9cQ1xe2x80x9d to be corrected from the supplied local target value xe2x80x9cTnxe2x80x9d and the condition xe2x80x9cbxe2x80x9d (packing density) of the output buffer 2010, and then supplies the calculated quantizing step Q1 to the quantizing means (not shown) employed in the encoder 2007. After the non-compression image has passed through the delay circuit, this non-compression image is supplied to the encoder 2007 so as to be encoded. At this time, the quantizing means provided in the encoder 2007 quantizes the non-compression image by using the quantizing step Q1. The encoder 2007 sequentially outputs the encoded bit stream to the buffer 2010.
As previously explained, the code converting apparatus 2000 can make the bit rate of the input bit stream equal to the bit rate of the output bit stream, while the pre-analyzer can be omitted, in such a manner that the bit stream is demultiplexed by the demultiplexing circuit 2003 and then the demultiplexed bit streams are entered to the target bit calculator 2009.
However, in the above-described conventional code converting apparatus 2000, when the demultiplexing circuit 2003 demultiplexes the input bit stream up to the macroblock layer, since this demultiplexing circuit 2003 requires the variable length decoding function, there are various problems that the circuit scale is increased and furthermore, the calculation amount is increased. There is another problem that the calculation amount required for performing the decoding operation by the decoder and also the calculation amount required for executing the encoding operation by the encoder are not reduced, as compared with such a case that the decoder is merely connected in series with the encoder. In addition, as indicated in FIG. 18, after such an input bit stream which has been once coded is completely decoded, this decoded input bit stream is again coded. As a result, there is a further problem that the deterioration occurs in the image quality.
The present invention has been made to solve the problems, and therefore, has an object to provide an image coding system converting apparatus, and an image coding system converting method, capable of converting an image coded by one image coding system into another image coding system in a high speed, while suppressing a deterioration of an image quality, and also to provide a recording medium for recording thereon a computer readable program capable of executing the image coding system converting method.
In the image coding system converting apparatus and the image coding system converting method according to the present invention, the separation means (step) separates the input data into the coded image data and the added information data, this image data is produced by coding a picture in the unit of a block, which owns a plurality of blocks constituted by a predetermined number of pixels, and the added information data contains data amount information of the picture; the decoding means (step) inverse-orthogonal-transforms the coded image data; the allocation means (step) allocates the upper limit data amount to the respective blocks owned by the picture based upon the data amount information of the picture; and the coding means (step) cuts the data of the subject block in such a case that the data amount of the subject block is larger than, or equal to the upper limit data amount set to the subject block, and inverse-orthogonal-transforms the cut data.
In the image coding system converting apparatus and the image coding system converting method of the present invention, the separation means (step) separates the input data into the coded image data and the added information data, this image data is produced by coding a picture in the unit of a block, which owns a plurality of blocks constituted by a predetermined number of pixels, and the added information data contains data amount information of the plurality of blocks; the decoding means (step) inverse-orthogonal-transforms the coded image data; the allocation means (step) sets the upper limit data amount of the respective blocks based upon the data amount information of the picture; and the coding means (step) cuts the data of the subject block in such a case that the data amount of the subject block is larger than, or equal to the upper limit data amount set to the subject block, and inverse-orthogonal-transforms the cut data.
As previously explained, in the decoding means (decoding step), only the inverse-orthogonal-transforming operation is performed while both the dequantizing operation and the inverse DCT transforming operation are not carried out, and in the coding means (coding step), only the orthogonal transforming operation is carried out while both the DCT transforming operation and the quantizing operation are not carried out. As a result, the calculation amount required for converting a certain image coding system into another image coding system can be largely reduced. Accordingly, it is possible to provide the image coding system converting apparatus, the image coding system converting method, and the recording medium, by which the highspeed image coding system converting operation can be realized. Also, since the data of the block is cut based upon the upper limit data amount, even when a certain image coding system is converted into another different image coding system, these image coding systems can be converted at the substantially same bit rate as the original bit rate without giving any large adverse influence to the image quality.
Further, according to the present invention, the separation means (step) separates the input data into the coded image data and the added information data, this image data is produced by coding a picture in the unit of a block, which owns a plurality of blocks constituted by a predetermined number of pixels, and the added information data contains coding information of the each block and quantizing parameter information; the block coding system judging means (step) judges the coding system of the subject block with reference to the coding information of the subject block; the decoding means (step) executes either the inverse-orthogonal-transform operation and the dequantizing operation with respect to the subject block in response to such a judgement result made of the block coding system judging means (step); the coding means (step) includes the quantizing parameter output means (step) for outputting a new quantizing parameter based upon the quantizing parameter information, wherein the first coding means (step) is performed which orthogonal-transforms such a block which has been inverse-orthogonal-transformed by the decoding means (step) without quantizing the block; and the coding means (step) wherein as to such a block which has been inverse-orthogonal-transformed and dequantized by the decoding step, after the new quantizing parameter outputted from the quantizing parameter output means (step) is multiplied by the DCT coefficient of the block, the second coding means (step) quantizes and also orthogonal-transforms the block.
More specifically, the block coding system judging means (step) judges as to whether the subject block corresponds to the intra-coded block, or the inter-coded block; and when the block coding system judging means (step) judges that the subject block corresponds to the intra-coded block, the decoding means (step) inverse-orthogonal-transforms the subject block judged as the intra-coded block, whereas when the block coding system judging means (step) judges that the subject block corresponds to the inter-coded block, the decoding means (step) inverse-orthogonal-transforms and also dequantizes the subject block judged as the inter-coded block.
As described above, since the quantized DCT coefficient is directly used for the intra-coded type block, both the dequantizing calculation required in the decoding means (decoding step), and also the quantizing calculation required in the coding means (coding step) can be omitted. Accordingly, the highspeed image coding system converting operation can be realized. Also, as to the inter-coded type block, the new quantizing parameter is produced by multiplying the new quantizing parameter by the DCT coefficient of this block, and this inter-coded type block is requantized by using this newquantizing parameter. Therefore, even when a certain image coding system is converted into another different image coding system, these image coding systems can be converted at the substantially same bit rate as the original bit rate without giving any large adverse influence to the image quality.
Also, the block coding system judging means (step) judges as to whether the subject block corresponds to the intra-coded block, or the inter-coded block; and when the block coding system judging means (step) judges that the subject block corresponds to the intra-coded block, the decoding means (step) inverse-orthogonal-transforms the subject block judged as the intra-coded block, whereas when the block coding system judging means (step) judges that the subject block corresponds to the inter-coded block, and further judges as to whether the coding system of the block belonging to the frame time-sequentially succeeded to the frame to which the subject block, and located at the same position of the subject block corresponds to the intra-coded block, or the inter-coded block; when the judged block corresponds to the intra-coded block, the decoding means (step) inverse-orthogonal-transforms the subject block judged as the intra-coded block, whereas when the judged block corresponds to the intra-coded block, the decoding means (step) inverse-orthogonal-transforms and also dequantizes the subject block judged as the inter-coded block.
As previously explained, the quantized DCT coefficient is directly employed with respect to both the intra-coded type block, and also such an inter-coded type block that the coded type of the block corresponds to the intra-coded type block, and this block is located at the same position as that of the subject block of such a frame time-sequentially subsequent to the frame to which the subject block belongs. As a result, the dequantizing calculation required in the decoding means (decoding step), and also the quantizing calculation required in the coding means (coding step) can be omitted. Accordingly, the more highspeed image coding system converting operation can be provided. Also, the blocks other than the above-explained block may readily cause the image qualities to be deteriorated while the blocks located at the same positions within the time-sequentially subsequent frame are such inter-coded type blocks. However, these blocks may become the intra-coded type blocks in future. Therefore, it is possible to realize such an image coding type converting operation while suppressing the deterioration in the image qualities. It should also be noted that since a total number of frames to be judged by the above-explained block coding system judging step is reduced, the time duration when the deterioration in the image quality occurs can be shortened.
Also, the added information contains bit amount information of each of the blocks; the block coding system judging means (step) judges as to whether the subject block corresponds to the intra-coded block, or the inter-coded block; and when the block coding system judging means (step) judges that the subject block corresponds to the intra-coded block, the decoding means (step) inverse-orthogonal-transforms the subject block judged as the intra-coded block, whereas when the block coding system judging means (step) judges that the subject block corresponds to the inter-coded block, the decoding means (step) either inverse-orthogonal-transforms or inverse-orthogonal-transforms and also dequantizes the subject block judged as the inter-coded block based upon an activity indicative of a degree of coding difficulty which is acquired by multiplying the bit amount of the subject block by the quantizing parameter.
As previously explained, since the activity is used, the simple calculation can be used to judge that the quantized DCT coefficient can be directly employed to which block among the inter-coded system block. As a result, the highspeed image coding system converting operation can be realized with smaller calculation amounts than those of the prior art.
Also, the transforming-coefficient calculating means (step) calculates the transforming coefficient based upon a difference in compression ratios before/after the image coding systems are converted; and in the quantizing parameter output means (step), when the transforming coefficient calculated by the transforming-coefficient calculating step is larger than 1, the quantizing parameter output means (step) sets the new quantizing parameter as the multiplier indicated by the quantizing parameter information; whereas when the transforming coefficient is smaller than 1, the quantizing parameter output step sets the new quantizing parameter as the divisor indicated by the quantizing parameter information and outputs the new quantizing parameter.
As a consequence, even when a certain image coding system is converted into another different image coding system having a different compression ratio from that of the first-mentioned image coding system, the quantizing error can be reduced. This quantizing error corresponds to a difference between the DCT coefficient dequantized by the original quantizing parameter and the DCT coefficient quantized by the new quantizing parameter. As a result, the image coding system converting operation can be realized while suppressing the deterioration in the image quality caused by the quantizing error related to the quantizing transform.
More specifically, the quantizing parameter output means (step) outputs the original DCT coefficient as the DCT coefficient as the block in such a case that while either the multiplier of the quantizing parameter or the divisor thereof indicated by the quantizing parameter information is used as the new quantizing parameter, the DCT coefficient obtained by multiplying the DCT coefficient of the block by the multiplier, or the divisor becomes smaller than the original DCT coefficient.
When the DCT coefficient obtained by multiplying the new quantizing parameter by the DCT coefficient of the block becomes smaller than the original DCT coefficient, the quantizing error will occur, resulting in the deterioration of the image quality. Therefore, such an image coding system converting operation can be realized while considerably suppressing the image quality deterioration caused by the quantizing errors related to the quantizing transform, in accordance with the above-explained converting apparatus, method, and recording medium.
Also, the transforming coefficient may be preferably equal to such a value obtained by dividing the compression ratio of the coding system before the coding system is converted by the compression ratio of the coding system after the coding system is converted.
Also, the added information data contains the data amount information of the picture; the upper limit data amount calculation means (step) for calculating the upper limit data amount of the picture based upon the data amount information of the picture; and the transforming coefficient may be preferably equal to such a value calculated by the ratio of the data amount indicated by the data amount information of the picture to the upper limit amount of the picture calculated by the upper limit data amount calculating means (step).
Also, the transforming coefficient is equal to a predetermined value which is previously determined in response to the quantizing parameter indicated by the quantizing parameter information every combination between the coding system before being converted and the coding system after being converted.
As previously explained, since the quantizing parameter corresponding to the transform coefficient is previously determined, the highspeed image coding system converting operation with a small calculation load can be realized.
Also, the quantizing parameter output means (step) determines the new quantizing parameter of the subject block based upon the quantizing parameters of a plurality of continuous blocks subsequent to the subject block with respect to the block which is inverse-orthogonal-transformed and dequantized by the decoding step.
As explained above, when the new quantizing parameter of the subject block is determined, the block subsequent to this subject block is prior-read so as to increase/decrease the optimum quantizing parameter. As a result, it is possible to suppress a total number of blocks in which the deterioration in the image quality will occur in connection with the transform of the quantizing parameter.
Furthermore, when both the image coding system converting apparatus and the image coding system converting method output the image data which has been coded in connection with the MPEG-4 standard, the quantizing parameter output means (step) reads the respective quantizing parameters of a plurality of continuous macroblocks with respect to a subject block which is inverse-orthogonal-transformed and dequantized by the decoding means (step) so as to determine a set region of adjoining macroblocks whose quantizing parameters are smaller than a predetermined value; and determines the quantizing parameter of the subject macroblock based upon the quantizing parameters of a plurality of continuous macroblocks subsequent to the subject macroblock within the set region.
In the MPEG-4 standard, there is such a limitation that a difference in the quantizing parameters between the adjoining macroblocks must be selected to be smaller than, or equal to xc2x12. As a result, when a new quantizing parameter is desirably selected to be larger than, or equal to xc2x13, such a set area of the adjoining macroblocks is determined, in which the difference in the quantizing parameters is smaller than a preselected value. Thus, the difference in the quantizing parameters among the macroblocks within this set area may be made smaller than, or equal to xc2x12. Accordingly, such a new quantizing parameter can be determined irrespective of the above-explained limitation. Therefore, it is possible to realize the image coding system converting operation with high flexibility.
Furthermore, a recording medium, according to the present invention, is featured by such a recording medium for recording thereon as a computer readable program, the image coding system converting method as mentioned above.